What 4X Games Actually DO Have Acceptable Diplomacy Systems?

[steveg700]

In reading about Humankind, I keep reading how diplomacy in Amplitudes games is lacking. So, this begs the question of where we can find a reliable bellwether.

What game do you think represents the best-in-class (if any), and what pushes it ahead of the pack?

[w@ke_]

It's only 4X-adjacent, but Crusader Kings almost plays like a diplomacy system with some spreadsheet-based combat and events stapled to it. Alliances actually grow, assert, and collapse in a relatively realistic manner. Although a lot of its diplomacy is based on the game's Sims-ish elements, which isn't everyone's cup of tea.

Star Ruler II has a cool card-based diplomacy system. Some people say it doesn't really qualify since it's so different than most 4X diplo systems, but IMO that's fine because the things it's different than are obsolete, unfun vestiges of terrible 90s diplomacy. For my money, making it its own sub-game with limited but really cool effects is way more fun than run of the mill spreadsheet diplomacy.

I really enjoy the diplomacy in Civ IV, but that's because of the opaque optimization mini-game that grows out of a really bad core system with all the faults I've just bemoaned. So obviously not the answer you're looking for.

[hurleybird]

A diplomacy system is well designed to the extent that it can maximise choice while minimising exploits. These two factors are almost always in tension.

And then you also have a theoretical vs. practical aspect. The trade table approach (Ala. Civ, GalCiv, and some others) allows almost infinite degrees of choice, and so is theoretically very good. It's an especially great choice for multiplayer. The problem is that creating an AI that can understand the nuances of such a system is fundamentally difficult, and I don't think anyone has been able to get a passing grade there just yet. Still, it's a good thing that some are trying.

Besides that, there are some obvious design sins that seem endemic to the genre:

• When a system is designed in a way that creates a schism between AIs and players. Eg. In certain games the AI will tell me whether or not it will accept a deal before I actually propose it, but the player is not similarly compelled. This is akin to playing Poker with X-Ray specs. The easy solution here is to just... not do that. Give the player some indication of whether his proposal is fair or not, but don't let him know the AI's decision ahead of time.
  
  
• A somewhat less obvious sin of the aforementioned variety is diplomacy modifiers. These also create a schism between players and AIs. They're worthless against players, who don't care about your +10 diplomacy modifier, and at the same time tie AIs hands behind their backs. While somewhat understandable in single player, these are downright heretical in multiplayer. The solution is to either not have them, or to have them also bind the player's actions in some way, or best yet to have diplomacy modifiers drive external factors, so that other players/AIs want to be friendly (or hostile) with you for actual concrete reasons -- better trade yields, morale, etc.
  
  
• Besides that, you'll often find systems that are somewhat fun when players use them poorly, exploitative when they use them properly, and downright fun destroying when AIs use them properly. Tech trading is a great example: I trade six techs with six other empires -- I get six techs while everyone else gets one. A proper AI should then try to trade techs as often as possible, but this would obviously make the game unfun, so they don't. For designers, the solution shouldn't be to tie the AIs hands behind their backs, but to fix any fundamental issues. So while you could just get rid of tech trading, a more interesting approach might be something like adding an opportunity cost to proposing a tech trade, but not for accepting someone else's proposal. Given proper balance, that might do the trick, and at a minimum will at least temper the usual exploits (This is actually what I'm trying in the next version of DG!)

So, I'm not sure if there's a "reliable bellwether," -- although I don't have an encyclopedic knowledge of 4X games to say that for certain -- but we can certainly look at what people are doing wrong and avoid the same mistakes. To a large extent, I think the same mistakes continue to be propagated primarily because designers are copying past approaches without really thinking through everything, so the search for a "reliable bellwether" may in and of itself be a large part of the problem.

[Gregorovitch]

Ck and EU have I would think the gold standard for diplo in strategy games. I would say main reasons are:

1. there are a crap ton of diplo options compared to most games all of which have meaningful effects and achieve objectives for both player and AI factions. There are also a crap ton of factions compared to most games. Crap ton squared.

2. The scores the diplo runs on a re transparent to both player and AI - everybody can see what everybody else is doing all the time (if they bother to look) and therefore both AI and player can make sensible and somewhat realistic decisions from that info.

3. Player and AI have a limited number of most diplo options they can do at once. Limitation is both by number of diplomats you are currently allowed and by limitations on number of certain agreements you can have live at once. This means info gleaned from 2 has more significance and reliability since choices made = clear indication of strategic objective. Cost of blind-siding/duplicity is high (I've no idea if AI does any of that sort of thing).

4. Sheer number of factions and options means it's pretty much impossible for player to keep track of everything that's going on. AI can, therefore it has one edge that most, or possibly all, other diplo AIs don't in the otherwise one-sided diplo game.

All in all these things add up to a secret sauce that can I think be summarised in three ways:

* it doesn't try to convince you, it tries to confuse you (and often succeeds)
* as a result it feels "real" 'cos you never know exactly what's going to happen
* you can manipulate some of the AI factions some of the time, but no way can you manipulate all of them all of the time.

I'd also add that Total War: Three Kingdoms has made a big advance in diplo AI, I think by common consent. Of course one could argue that's not saying much, but the improvement is pretty big IMO and augers well for the franchise's future.

I think the big reasons for this are:

* Both factions and player have a strong reason to make friends to trade with because armies cost a fortune to maintain. Roughly 3 or 4 trade agreement = one extra stack.

* Both factions and player have a strong reason to make friends because friends won't attack them (usually) so they can attack someone else in relative safety. Non-friends will very likely attack an exposed flank if at war with someone else.

* The character trait system is comprehensive and meaningful in the way AI factions conduct diplomacy. Faction leaders will behave and react according to these traits (often emotionally as much as from rational calculation), and this feels real. Like EU/CK, these traits are transparent, visible to both player and AI.

* Also like EU/CK relationships between factions are scored between about -400 to +400 (although usually within the -200 to +200 range) and trend over the long term for fairly obvious and credible long term reasons.

* Trade deals can involve many things weighed in the balance (money, money over time, territory, food (food is incredibly important), ancillaries and items, marriages, vassallage/protection, various treaties etc etc. A trade deal has a numeric value as to whether it's good enough or not to accept, zero being the minimum to accept. Every item in the deal has a specific + or - value added to this score. Again like EU/CK all this is transparent. If you make a good deal for them, greater than zero in their favour basically, you get an ongoing relationship boost for it.

* The better your personal relationship, as well as the more strategic value your friendship has, the better deals you get and the more likely your friend will sign treaties with you, agree to join a war with you etc. Much better by a very wide margin. This means cultivating a good long term relationships can be very advantageous. It's about trust, honour, fair dealing, money and military cooperation.

* There are two alliance systems, the feudal vassal system and the coalition/treaty system, running side by side. The financial cost of being a vassal is not actually that big, especially in view of the enormous degree of protection it affords in some situations. The main down side is being instructed to go to war against X whether you like it not. This makes for an interesting juxtaposition, enlivening the diplo game considerably.

* There are a lot of factions. Not anything like as many as EU/CK obviously, but still a lot and I would say the intensity and frequency of diplo interaction between factions is considerably greater than in EU/CK which goes some way to make up for that.

It all adds up to a pretty rich, believable and complex diplo environment, and I think the three ingredients of EU/CK's secret sauce are applicable to TW:3K too, just not quite to the same extent.

[PengoPaddy]

I liked Civ IV's diplomacy because to some degree the AI's didn't play as 100% independent players but rather had modifiers based on things like religion of government type (something like that) which meant they were part of an easily understood mechanic that the player could choose to engage with to produce known results. If I'm playing single player against the AI I don't need them to be ruthless to enjoy myself, I just need them to provide something to interact with to get ahead (one way or another) in the game.

Agree with the above post re EU4, the AI's ability in that game to form alliances to counter a singular threat is fantastic.

[Martok]

If one were to add "diplomacy systems where the AI actually behaves somewhat sensibly", then I would add Stellaris and Armada 2526 to the list (along with seconding the nod for CK2).

[hurleybird]

Originally posted by Martok:

If one were to add "diplomacy systems where the AI actually behaves somewhat sensibly"

One thing I forgot to say before is that there's also a fair amount of tension between "The AI is playing sensibly" and "The AI is playing appropriate to their in-universe role." To the extent that your design is tight, you may be able to intersect both dimensions so that good role playing has the tendency to also be strong playing. Eg. Empire A's lore states that they are good at X, and because they're so good at X, having their AI focus on X fits thematically while also being a strong tactic.

[Martok]

An extremely good point, hurleybird.

There is that "sweet spot" in AI behavior (which you just described) that I really enjoy seeing, but it seems to be quite rare. I imagine coding AI to do both things -- play the game well, yet also "roleplay" it well -- is probably just a tad difficult.

[Marfig]

I sometimes wonder what game are you making, hurleybird. Will it be so good as you make it sound like? And I don't want to be mean. It is an honest question. Because in the end this is not about what can be done, but how and if it can be done. The amount of resources we have at our disposal and if we actually have the know-how to pull it off.

Because we all have our ideas about what constitutes a good AI, I am even reading words like "golden standard". But commercial strategy AIs are really as bad as it can get and this suggests that resources and know-how are not enough and technology isn't just there yet. It's interesting that this thread starts by calling it bad, what is essentially one of the best 4x AIs in the market, and then asks what are the good AIs. Well, @steveg700, you can't get much better 4x AI than EL and ES. And that is not saying much.

And please do not confuse how an AI is configured with what it is actually capable of doing. When a modder took on himself to change the AI files in Endless Legend, he created the AI+ mod. If you play the game with this mod, you will know why I am calling it one of the best 4x AIs out there. No asset or code modifications. Just plain AI configuration went into it.

But even then, 4x AIs have a very short lifespan. If I were to generalise, around the 100th hour mark we start gaining a deep enough understanding of a game that the AI starts becoming predictable. That is when (especially of you specialise in a faction or two) you start winning by default. There is no way around it. You anticipate the AI on basically all of the components of the game, from combat to economics and diplomacy. That's the "golden standard" of 4x AIs. And it is depressing.

And yes, CKII is no different. I suspect one of the reasons the AI is well regarded is simply the fact the game plays very different to anything else in the market, and the AI looks more abstract by proxy. Yet, Reddit, Paradox and other CKII communities are riddled with comments (by actual players) about how bad the AI really is.

And that is the only logic I employ to evaluate an AI -- i.e. how may gameplay hours before I start steamrolling the map with only few surprises and without much effort. Anyone can call it reductive, if they want. But not only this is a good way to evaluate the commercial benefit of my purchase (because this is when I usually stop playing a strategy game), this is also the only indicator that matters when evaluating AIs. It is akin to the Turing Test -- i.e. how many questions I can make before I realise this is an artificial intelligence.

If I approach my 4x gaming casually however, I will have plenty of games that offer a challenge before I start challenging the game. But for those of us that like to explore deep into the games we play, the poor state of not just 4x, but all strategy AI, is very evident. And the answer to which ones are good is an empty set.

[cadfan17]

This is why asymmetrical single player 4x design is so underrated. “Build a 4x AI that mimics a human” is a losing proposition. “Build a 4x where the AI doesn’t have to mimic a human to be interesting” is not.

There are loads of game genres that do this. It’s unfortunate that 4x specifically is so hidebound that it won’t.

If people played more board games they’d know this.

Even the very core of a 4x is nearly (actually?) unattainable for AI in the typically designed 4x.

Imagine a simple game. Eight players. Each turn each player chooses another player to punch. Players can talk. You can make promises but the only enforcement is who you choose to punch. Last man standing wins.

To play this strategically you would want to convince people that you’re really hurt, and that they should punch someone else. In particular they should punch whoever is doing the best. This is all straight game theory. If you end up in a 1v1 with the toughest and least hurt guy you lose, so it is in everyone’s interest to gang up on him. But once you’ve all hit him a few times he might not be the biggest threat. If you think he’s been knocked down to where you could win a 1v1 you want to leave him there, and get everyone to go after the next most dangerous guy. And if that’s you, you want to convince everyone it isn’t.

There are SO MANY board games that boil down to this. Munchkin is the easiest and most well known example. Or Catan.

And every multiplayer 4x ever written.

But making an AI script for this is insanely hard. The bluffing part is insanely difficult. The promises part is insanely difficult. Threat assessment is sometimes easy, except that to mimic human judgment you need to make faulty threat assessment that can be fooled in understandable ways, but without being predictable in what fools it! And that’s hard.

Humans play a game like this automatically. Children figure it out.

AI? Ehhhh...

The solution to good 4x programming is to avoid making your game run on a system that AI can’t do.

[hurleybird]

Originally posted by cadfan17:

This is why asymmetrical single player 4x design is so underrated. “Build a 4x AI that mimics a human” is a losing proposition. “Build a 4x where the AI doesn’t have to mimic a human to be interesting” is not.

There are loads of game genres that do this. It’s unfortunate that 4x specifically is so hidebound that it won’t.

If people played more board games they’d know this.

Even the very core of a 4x is nearly (actually?) unattainable for AI in the typically designed 4x.

Imagine a simple game. Eight players. Each turn each player chooses another player to punch. Players can talk. You can make promises but the only enforcement is who you choose to punch. Last man standing wins.

To play this strategically you would want to convince people that you’re really hurt, and that they should punch someone else. In particular they should punch whoever is doing the best. This is all straight game theory. If you end up in a 1v1 with the toughest and least hurt guy you lose, so it is in everyone’s interest to gang up on him. But once you’ve all hit him a few times he might not be the biggest threat. If you think he’s been knocked down to where you could win a 1v1 you want to leave him there, and get everyone to go after the next most dangerous guy. And if that’s you, you want to convince everyone it isn’t.

There are SO MANY board games that boil down to this. Munchkin is the easiest and most well known example. Or Catan.

And every multiplayer 4x ever written.

But making an AI script for this is insanely hard. The bluffing part is insanely difficult. The promises part is insanely difficult. Threat assessment is sometimes easy, except that to mimic human judgment you need to make faulty threat assessment that can be fooled in understandable ways, but without being predictable in what fools it! And that’s hard.

Humans play a game like this automatically. Children figure it out.

I partially agree with you in principle, bluffing and certain other dimensions (like trustworthiness) are difficult problems for AIs. Your specific example is poor, since few 4X games are known to expose bluffing as either an explicit mechanic or even an implicit component. In fact, many 4X games expose the strength of players across multiple dimensions without error bars (MoO 1 and 2 come to mind, but this is a very common practice in general), making it impossible to, say, pretend to have a smaller fleet than you actually do.

The ability to bluff is an idea that's been rolling around in my head for Dominus Galaxia. Specifically, in the soon-to-be-released next version, you still have a ranking of empire strength across multiple dimensions, but you also have error bars that shrink based on the amount of known information and the strength of your spy networks. At one time, I was thinking about being able to spread disinformation so as to look more or less powerful than you actually are, and it's possible I could come back to that, but the issue was that while there were many good reasons to appear stronger, the same was not true of appearing weaker. And of those few reasons, none would be straightforward to teach the AI.

Now, where I disagree with you in principle is that strategy games should be designed in ways that AIs don't need to try to mimic humans in the first place. Do that, and you no longer have a strategy game. You have something else entirely. It may or may not be good, but you aren't talking about a strategy game let alone a 4X.

Lastly, the point about asymmetry is something of a red herring. The degree of symmetry is orthogonal to how difficult it is for an AI to perform its task. Eg. Wink murder is asymmetric and hard. Tic-tac-toe is symmetric and easy.

Originally posted by cadfan17:

The solution to good 4x programming is to avoid making your game run on a system that AI can’t do.

There's no such thing as systems that AIs can't do, only systems that are difficult. The problem is that there's a massive correlation between "difficult for AI to do" and "strategically deep." The corollary also holds. For example, math puzzles, memorisation tests, and reaction speed aren't at all strategic, but AIs have fundamental advantages. You could sum things up as: strategic systems are hard (for AIs), but not all hard systems are strategic. Easy systems are rarely strategic, but not all non-strategic system are easy.

Anyway, as a designer you should perform a cost-benefit analysis on a case-by-case basis. Good design can help nudge things in the direction of a better strategic-depth-to-AI-difficulty ratio, but at the end of the day the better strategy game you make, the more likely it is that it will be difficult for an AI to play your game.

That said, while I think that it's obvious that a strategically empty 4X is fundamentally poor (or at least is specifically poor as a 4X strategy game), it's less obvious that a (theoretically at least) strategically pure 4X game is fundamentally better than one that is less pure. My thoughts run along the lines that, everything else being equal a more strategically deep game is preferable, but the further you go in that direction the more tension you generate across various competing dimensions, making it difficult to keep everything equal in the first place. TNSTAAFL.

[hurleybird]

Originally posted by Marfig:

I sometimes wonder what game are you making, hurleybird. Will it be so good as you make it sound like? And I don't want to be mean. It is an honest question. Because in the end this is not about what can be done, but how and if it can be done. The amount of resources we have at our disposal and if we actually have the know-how to pull it off.

Maybe I should reply to this in the Dominus Galaxia thread sometime later. Don't want to derail this one too badly ;)

[Troy]

I'm not seeing many 4X games actually listed here. Maybe there isn't such a thing as a 4X game with good diplomacy. Maybe 4X games just don't have that due to their very nature.

[w@ke_]

Originally posted by Troy_Costisick:

I'm not seeing many 4X games actually listed here. Maybe there isn't such a thing as a 4X game with good diplomacy. Maybe 4X games just don't have that due to their very nature.

I think it's more due to the very limited creative space 4X game designers seem to be interested in exploring. Diplomatic interactions usually just amount to summing some totals and seeing if the offer will be accepted, just like they were 25 years ago. There's seldom anything like jockeying or strong-arming that could really make it feel like a push-and-pull/give-and-take affair (although Civ V and VI did take some baby steps in that direction).

If people give that design space an honest shot at more engaging gameplay and find it just isn't doable for whatever reason, they could do a Star Ruler and make diplomacy into a power-based system. Make diplomatic machinations and maneuverings tie into in-game resources and assets so that they're a part of the game's core mechanisms instead of a spreadsheet. This is something I've held space-based 4X designers should do with fleet and troop tactics for years, but I think it could bear some spicy fruit in the diplomatic sphere as well. IMO that also could help with the gameplay/story integration hurleybird points out as a challenge.

TBH a big piece of the baggage with diplo in 4X if we go way back is that 4X is largely an offshoot of board gaming. Which in this case is to its detriment, since negotiations that would work well in board games ("I will give you # of X for # of Y" or "I will not attack your western flank if you let me take control of this city") actually are pretty interesting in a social game with many people party to the conversation and the greater negotiation framework. But it feels like 4X diplomacy is today is pretty much a copy-paste of framework, and those same transactions with a computer just don't have any teeth. It's up to the designers to either come up with ways of emulating that layer (maybe possible, maybe not), or find new systems that can create interesting gameplay moments. Or we can just keep having terrible diplomacy forever, I guess.

Or maybe I'm just talking out of my ass. :P

[Bou]

Space Empires V got potential (huge selection of diplomatic options which can be combined into customized treaties + back and forth negotiation with the AI about treaty content), but the problem is that the AI doesn't make use of the majority of diplomatic options. Captain Kwok, creator of the balance mod, is working entirely on AI diplomacy at the moment. The new update will be released at the end of this year. I will check it out once it has been released and report back.

[cadfan17]

Hurleybird- A game doesn’t need an explicit bluffing system for bluffing to be a factor if the game has kingmaker mechanics. All it needs is any ambiguity at all plus humans who communicate. Risk players have nearly perfect information. But bluffing about who is “in the lead” is still present because people may estimate probabilities differently, have different risk tolerances, or may make different predictions about other player behavior.

The reason this is insanely hard for AI is because the reasoning is iterative.

If you are in the best position to win a 1v1 with all other players you seem like you are in the lead and we should hang up on you. But once we agree to do that then maybe the player who is really in the lead, subject to our expectations about the future, is the player best positioned to be the strongest 1v1er once your abilities are degraded by our aggression. So it makes sense to defect from the alliance against you to address your expected successor. But if we are confident that alliances will wipe out the two leaders, maybe the third guy is the real danger and maybe we are just helping him by going after the top two. But if we keep defecting from the primary alliance to counter the next risk then maybe the first guy will win anyways.

Meaningful predictions require estimations of the game state in the future under multiple conditional alternate scenarios, factoring in the likely results of other players running the same calculations and changing their behavior.

It’s a nightmare and human brains do it intuitively because we are evolved social animals.