Generate subtitles with live caption

[pizza4life]

There’s a Microsoft Live Caption feature on Windows that takes in audio (from videos, games, live stream, audio files etc) and then generate captions in real time, which work as subtitles. You can use it on games without subtitles like Assassin’s Creed 1. It is not perfect but better than nothing, especially for folks who are hard at hearing. And we live in the reality where not all games support all sorts of languages in the world

I think it will be great if Steam bring this further, and have a feature that overlay live caption in real time for games. Or even better, output the live caption to any language too. This will not only aid folks who have hearing issues, but also allow players to play games of foreign languages. A game with only Japanese language support can have English or Chinese live caption. Additionally players who mute their audio can use the live caption/subtitles too, or a temporary workaround when the audio speakers broke etc

[pizza4life]

Yah it’ll be great if it could translate on screen menu text and such, but it’ll require Steam to scan and read the game visual for texts

[HikariLight]

You do know Live Caption is either a person or a bot translating the words in real time right?
As Valve will not have staff waste their time live translating stuff for players, the only option is a bot and those may not translate things correctly.

Also if it's not a game developed by Valve, it is not Steam's concern.

[KalGimpa]

it could be a neat feature

i will say that my lady and i spend almost as much time laughing at the subtitles from these

as we do watching the show

[pizza4life]

Originally posted by HikariLight:

You do know Live Caption is either a person or a bot translating the words in real time right?
As Valve will not have staff waste their time live translating stuff for players, the only option is a bot and those may not translate things correctly.

Also if it's not a game developed by Valve, it is not Steam's concern.

I am actually suggesting Valve to develop the feature which use AI bot to make the real time translation, which can then be used across all games. Not translating things accurately is understandable and better than nothing, and the translation accuracy can get better over time.

[Satoru]

Originally posted by pizza4life:

Originally posted by HikariLight:

You do know Live Caption is either a person or a bot translating the words in real time right?
As Valve will not have staff waste their time live translating stuff for players, the only option is a bot and those may not translate things correctly.

Also if it's not a game developed by Valve, it is not Steam's concern.

I am actually suggesting Valve to develop the feature which use AI bot to make the real time translation, which can then be used across all games. Not translating things accurately is understandable and better than nothing, and the translation accuracy can get better over time.

Note that this i pretty bad given that genreally speaking these thing don't work that good and are especially not good if the audio is not clear. Even Live Caption tells you that it can't do songs very well. Its not going to be able to caption a video game reliably

A much better solution is to advocate gamedevs to add captions to their games.

[pizza4life]

Originally posted by Satoru:

Originally posted by pizza4life:

I am actually suggesting Valve to develop the feature which use AI bot to make the real time translation, which can then be used across all games. Not translating things accurately is understandable and better than nothing, and the translation accuracy can get better over time.

Note that this i pretty bad given that genreally speaking these thing don't work that good and are especially not good if the audio is not clear. Even Live Caption tells you that it can't do songs very well. Its not going to be able to caption a video game reliably

A much better solution is to advocate gamedevs to add captions to their games.

Like I said, this is a “better than nothing” feature and it can only improve over time. Advocating game devs to add captions is only a better solution if they end up doing so. We’re living in the real world where it is not feasible for numerous reasons, whether it is the publisher going bust, financial incapable to add additional language support, moved on to next project etc.

This is similar to advocating game devs to add support for various game controllers, ultrawide screen, gyro, Mac, Linux or Steam Deck support etc. as much as it’ll be great if each and every Steam games do so, it is not realistic to expect that to actually happen. But Valve did put in work to improve that with SteamInput and Proton compatibility layer. Not perfect but nobody’s complaining