Interesting stuff with AI?

[Yskas]

So these days everyone is AI crazy, anyone know of any cool mods / projects there are around ARMA incorporating AI tech?

Remember seeing a mod on ARMA 3 that allowed you to command a Squad etc using your voice, but that leveraged Windows built in voice stuff, wonder if you could do something like that using AI tools or something?

I mean, apart from regular stuff like texture work, it'd be cool to see friendly/opfor AI intellect being boosted.

[CJ]

sign me up

[CJ]

im working on this of AI reinforcement learning that could be integrated into the ARMA Reforger mod. This example will focus on improving AI decision-making in combat situations. Here's a simplified reinforcement learning system using Q-learning:

This reinforcement learning system uses Q-learning, a popular algorithm in reinforcement learning. Here's a breakdown of how it works:

The system maintains a Q-table (`_qTable`) that stores the expected rewards (Q-values) for each state-action pair.

The state is determined by the AI unit's health, ammo, and nearby enemies.

Actions include "attack", "defend", "flank", and "retreat".

The system uses an epsilon-greedy policy to balance exploration (trying new actions) and exploitation (using known good actions).

After each action, the system calculates a reward based on the outcome and updates the Q-value for that state-action pair.

Over time, the AI learns which actions are most beneficial in different situations.

To integrate this into your mod:

Call `fnc_initAILearning` for each AI unit you want to apply reinforcement learning to.

Implement the action functions (`fnc_performAttack`, `fnc_performDefend`, etc.) to make the AI execute the chosen actions.

Implement the reward calculation functions (`fnc_enemyKilled`, `fnc_tookDamage`, etc.) to evaluate the outcomes of actions.

Adjust the learning parameters (`_learningRate`, `_discountFactor`, `_explorationRate`) to fine-tune the learning process.

Consider persisting the Q-table between game sessions to allow for long-term learning.

This system will allow us AI to adapt its behavior based on the outcomes of its actions, potentially leading to more dynamic and challenging opponents. Reinforcement learning can take time to converge on optimal strategies, so I may need to run many iterations or pre-train the AI before deploying it in a mod. and with your's commanding a Squad etc using your voice would be out of this world lol.

[Yskas]

Originally posted by CJ:

im working on this of AI reinforcement learning that could be integrated into the ARMA Reforger mod. This example will focus on improving AI decision-making in combat situations. Here's a simplified reinforcement learning system using Q-learning:

This reinforcement learning system uses Q-learning, a popular algorithm in reinforcement learning. Here's a breakdown of how it works:

The system maintains a Q-table (`_qTable`) that stores the expected rewards (Q-values) for each state-action pair.

The state is determined by the AI unit's health, ammo, and nearby enemies.

Actions include "attack", "defend", "flank", and "retreat".

The system uses an epsilon-greedy policy to balance exploration (trying new actions) and exploitation (using known good actions).

After each action, the system calculates a reward based on the outcome and updates the Q-value for that state-action pair.

Over time, the AI learns which actions are most beneficial in different situations.

To integrate this into your mod:

Call `fnc_initAILearning` for each AI unit you want to apply reinforcement learning to.

Implement the action functions (`fnc_performAttack`, `fnc_performDefend`, etc.) to make the AI execute the chosen actions.

Implement the reward calculation functions (`fnc_enemyKilled`, `fnc_tookDamage`, etc.) to evaluate the outcomes of actions.

Adjust the learning parameters (`_learningRate`, `_discountFactor`, `_explorationRate`) to fine-tune the learning process.

Consider persisting the Q-table between game sessions to allow for long-term learning.

This system will allow us AI to adapt its behavior based on the outcomes of its actions, potentially leading to more dynamic and challenging opponents. Reinforcement learning can take time to converge on optimal strategies, so I may need to run many iterations or pre-train the AI before deploying it in a mod. and with your's commanding a Squad etc using your voice would be out of this world lol.

YES dude, this is the way, we're still in the dark ages when it comes to NPC AI, by all accounts the industry has regressed! The only use of AI I see in gaming is to save cost in textures and elbow grease jobs, not a peep about using it where it actually freaking matters, gameplay AI. I bet we'll see Arma modders spearhead this stuff, it's a great modding platform.

The voice command thing is neat as hell, there's a dude that made a mod for Arma 3 that utilised Windows built in voice service, but I think you'd want a platform agnostic solution to really make an impact, that's going to be tricky but hell with AI these days it seems anything is possible. And this being a milsim game you don't really have a huge scope of commands you're dealing with, so that's an upside at least.

Bohemia should encourage this bleeding edge AI modding and make their play data public so people can train models off it.

EDIT: Made a request to them to both make trainable data available and maybe get a basic in-game voice command function going so modders can hook it up to any NPC AI mods (I think the voice command thing will play a big part in the near future.)

Just imagine it, you're on a 16 player server, all players are squad leaders, leading realistic, experienced (trained) AI soldiers and assets, all naturally responding to voice commands. Bro this could be bigger than the "Day-Z" effect that propelled Arma into the mainstream back in the day.

[Yskas]

Neat video on the topic.

https://www.youtube.com/watch?v=dm_yY-hddvE

[ZigZack]

Honestly, the best approach for advanced AI is something along the lines of Utility AI (ie where a particular score results in a behavior). Mixing Utility with Behavior Tree would likely be optimal.

KAI utilized a scoring method for context.

[Chairman Mao's Chicken]

Originally posted by Yskas:

Originally posted by CJ:

im working on this of AI reinforcement learning that could be integrated into the ARMA Reforger mod. This example will focus on improving AI decision-making in combat situations. Here's a simplified reinforcement learning system using Q-learning:

This reinforcement learning system uses Q-learning, a popular algorithm in reinforcement learning. Here's a breakdown of how it works:

The system maintains a Q-table (`_qTable`) that stores the expected rewards (Q-values) for each state-action pair.

The state is determined by the AI unit's health, ammo, and nearby enemies.

Actions include "attack", "defend", "flank", and "retreat".

The system uses an epsilon-greedy policy to balance exploration (trying new actions) and exploitation (using known good actions).

After each action, the system calculates a reward based on the outcome and updates the Q-value for that state-action pair.

Over time, the AI learns which actions are most beneficial in different situations.

To integrate this into your mod:

Call `fnc_initAILearning` for each AI unit you want to apply reinforcement learning to.

Implement the action functions (`fnc_performAttack`, `fnc_performDefend`, etc.) to make the AI execute the chosen actions.

Implement the reward calculation functions (`fnc_enemyKilled`, `fnc_tookDamage`, etc.) to evaluate the outcomes of actions.

Adjust the learning parameters (`_learningRate`, `_discountFactor`, `_explorationRate`) to fine-tune the learning process.

Consider persisting the Q-table between game sessions to allow for long-term learning.

This system will allow us AI to adapt its behavior based on the outcomes of its actions, potentially leading to more dynamic and challenging opponents. Reinforcement learning can take time to converge on optimal strategies, so I may need to run many iterations or pre-train the AI before deploying it in a mod. and with your's commanding a Squad etc using your voice would be out of this world lol.

YES dude, this is the way, we're still in the dark ages when it comes to NPC AI, by all accounts the industry has regressed! The only use of AI I see in gaming is to save cost in textures and elbow grease jobs, not a peep about using it where it actually freaking matters, gameplay AI. I bet we'll see Arma modders spearhead this stuff, it's a great modding platform.

The voice command thing is neat as hell, there's a dude that made a mod for Arma 3 that utilised Windows built in voice service, but I think you'd want a platform agnostic solution to really make an impact, that's going to be tricky but hell with AI these days it seems anything is possible. And this being a milsim game you don't really have a huge scope of commands you're dealing with, so that's an upside at least.

Bohemia should encourage this bleeding edge AI modding and make their play data public so people can train models off it.

EDIT: Made a request to them to both make trainable data available and maybe get a basic in-game voice command function going so modders can hook it up to any NPC AI mods (I think the voice command thing will play a big part in the near future.)

Just imagine it, you're on a 16 player server, all players are squad leaders, leading realistic, experienced (trained) AI soldiers and assets, all naturally responding to voice commands. Bro this could be bigger than the "Day-Z" effect that propelled Arma into the mainstream back in the day.

What's funny is there have been games like what you are describing 20 years ago. They were janky, but at least developers were trying new things back then. The Black Hawk Down videogame on the original Xbox for example has ai voice commands if you get the microphone.

[Rick James Bish]

DCS + VoiceAttack does something similar. I can call out commands to AWACs, Flight, Ground Crew, etc.. and they respond. Of course it is predetermined responses but it adds a whole new depth since I have to use the cockpit radio and be on the correct frequency, etc.. I could see VoiceAttack being able to do the same for Reforger if the developer decided to support it.