Video textures

[butcho]

Hey,

yesterday I started building two scenes to test some video texture stuff. I created two 1080p image sequences (same animation, only a different lightning setup so I can blend them over) and used a script on the opacity setting. While doing that I noticed that the two video sequences always start to run out of sync after a while. Sometimes directly after the start of the wallpaper, sometimes it takes time until it happens. So I tried different encoded videos with h.264 and h.265, different profiles and quality settings, bitrates, etc. but no luck.

Don't know if this is a limitation of video textures but I just wanted to let you guys know, maybe it's fixable. Just for fun I tested the impact on my GPU while running five or six instances of my media player with a FullHD video and my GPU is only at 25-35% load so I guess the hardware is capable of running two or more video textures at the same time.

I can send you the project files if you might want to take a look. Might be helpful to see what I'm talking about. ^^

Thanks for your help.

btw: I also noticed that I got a lot of artifacts when using h.265 videos...but that's maybe because it's still in a Beta state.

[butcho]

Ah...forgot to add that these are 1080p 60fps videos. Maybe I should aim for 30 fps. Gonna check that out as well. Might be some dropped frames in WPE that cause this.

[Biohazard]

The video system of Windows doesn't support any proper synchronization so as a workaround we implemented a script callback that is fired when a video finishes which you can use to restart all videos at the same time (or just the other videos that are supposed to be synchronized).

Keep in mind such a callback is fired "after" a video finishes so you will likely have to reset the other videos with a little bit of time in advance. But in theory this should allow manual synchronization of clips, at least periodically each time a video finishes.

Originally posted by butcho:

btw: I also noticed that I got a lot of artifacts when using h.265 videos...but that's maybe because it's still in a Beta state.

This can also happen with normal video wallpapers and I think that it's unable to buffer the data quickly enough or that the program doesn't get enough CPU time. If you have this issue with 60 FPS, try using 30 FPS instead.

[butcho]

Originally posted by Biohazard:

The video system of Windows doesn't support any proper synchronization so as a workaround we implemented a script callback that is fired when a video finishes which you can use to restart all videos at the same time (or just the other videos that are supposed to be synchronized).

I this something WPE does automatically after the texture restarts or do I have to add this script manually?

This can also happen with normal video wallpapers and I think that it's unable to buffer the data quickly enough or that the program doesn't get enough CPU time. If you have this issue with 60 FPS, try using 30 FPS instead.

I switched to h.264 and then the artifacts are nearly non existent. I'm good with that. ^^

[Biohazard]

You'll need to make a script, something like this in the init function:

thisLayer.getVideoTexture().addEndedCallback(()=>{
thisScene.getLayer('OTHER VIDEO').getVideoTexture().setCurrentTime(0);
});

If setCurrentTime leads to artifacts you can also try stopping and playing the video instead.

[butcho]

Ok, understood. I'm trying that out. Thank you.

[butcho]

Seems like it's working. I figured out it was also some misconfiguration in my post processing tool. Most of the time when doing animations I often skip every other frame in a render to safe a lot of time and let the motion estimation algorithm put the missing frames in it. Since I could not get it to work I checked the video files again and discovered that one had a slight difference in length. Only a few ms but it was enough to screw the blending.

Keep in mind such a callback is fired "after" a video finishes so you will likely have to reset the other videos with a little bit of time in advance. But in theory this should allow manual synchronization of clips, at least periodically each time a video finishes.

Hope you don't mind my asking but the first run of the two video textures, after the wallpaper starts, is still a bit off sometimes and after that they run in sync. This is because of your little script I applied? Can I somehow apply it in a way that they run syncronised right from the start?

Sorry if this is a stupid question but I'm interested in understanding what's happening.

Thanks for your help.

[Biohazard]

In any init function, use setTimeout ( https://wallpaper-engine.fandom.com/wiki/SceneScript_Class_IEngine ) with a short delay and then call setCurrentTime(0) for every video.

I think it's to be expected that they may not be perfectly in sync until everything is loaded, because the videos take a moment to start up. It's really dependent on the system and how heavy the videos are in the end.

[butcho]

Ok, hope I get this right: the setTimeout callback delays the start of the videos for x ms so all videos are loaded and then your script from above keeps them running in sync right from the start of the wallpaper?

Can you please tell me where to put setTimeout in the init function? Tried it but it wasn't working. This is sometimes confusing to me because of some missing basic knowledge. I just don't know where to put what. Read some JavaScript sites about basic knowledge but that didn't help either.

Would appreciate the help and thanks for your time.

[Biohazard]

You probably forgot the "engine." part because it's part of the engine object, so try it like "engine.setTimeout" in the init function.