Development of a Spanish dubbing using Artificial Intelligence

[Juan carlos bodoque]

This week an idea came to me, what if I create a dubbing of Baldur's in Spanish using AI? So, I start to work. The plan is to take the audio files and introduce them to the AI called Whisper, which will transcribe the audios into text. Then, give those texts to Chat GPT and ask it to perform a translation, and finally introduce the English dubbing audio files and the already translated game dialogue text into the AI from elevenlabs to generate new audio files in Spanish.

Here's a link to YouTube with a short audio of my progress in this adventure of creating a Spanish dubbing with AI:

https://youtu.be/dkbyOj7criw

This week an idea came to me, what if I create a dubbing of Baldur's in Spanish using AI? So, I got to work.

It's not much, but it's a first try.

Of course, nothing is ever easy. So far, I've encountered several issues, but I made significant progress. First, finding the game audio files took me hours (since I have a very limited understanding of programming and an especially limited knowledge of video game development). This is because the audio files are packaged in a file called voice.pak. After hours, I managed to find a tool capable of unpacking the file. However, another problem arose; the audio files are in a format I had never seen before. The format is called WEM, meaning the files have a .wem extension. After even more hours of research, I managed to find a tool capable of converting the WEM format files to the OGG format, which the Windows player can play. But, there's a major problem. As I mentioned, I know almost nothing about programming, and the tool to convert the files is programmed in C#. It took me two days to understand how to use the tool. To better explain the next part, I should clarify that the audio files are basically divided into two folders: one called Localization and another called Mods. The former only contains very short audios and, compared to the other folder, doesn't have many files, weighing only 118 MB, while the folder called Mods weighs 4.77 GB. Today, I finally managed to convert all the files from the Mods folder to the OGG format. However, I couldn't do the same for almost any of the files from the other folder; the program throws an error. For now, that's all the progress I've made in 3 days.

If you're interested in the topic or want more details about my project to dub the video game in Spanish, or if you want to support me on my journey, reply or send a message to my Steam account. I'll update my progress every week.