Is it possible (in the future) to use RTX Tensor Cores for frame generation?

[Hakisak]

I'm loving this software.
Is it possible to utilize RTX Tensor Cores to improve efficacy of the Frame Generation and reduce rasterize utilization? This would be useful for games without raytracing/dlss so they wouldn't need to reduce their rasterization performance in order to run frame generation, increasing fps before frame generation making for a smoother experience.

Would this only be possible for RTX4000 cards? like how DLSS-FG is only for RTX4000?

Would there be any visual improvements?

-With my 4090 I see about 30w of gpu power increase with LS-FG-2.0 and gpu utilization increase of %12 (game locked at 60fps on a 175hz monitor). full pc specs in profile. Those numbers aren't bad, I think LS-FG is awesome, I'm just excited for future (software) technology that can utilize pc hardware to its fullest and more efficient.

Can't wait for the new update today/tomorrow!!

[ogioto]

Already discussed previously- using these core won't make the visuals better due to the indirect way LS captures the frames. So, the tensor cores wouldn't be too much of a help in that case.
Also, the dev is against adding options that can be used only by specific users, which also adds against making such option.

[lyndonguitar]

Originally posted by ogioto:

Already discussed previously- using these core won't make the visuals better due to the indirect way LS captures the frames. So, the tensor cores wouldn't be too much of a help in that case.
Also, the dev is against adding options that can be used only by specific users, which also adds against making such option.

I believe the aim of this suggestion isn't to "improve visuals", but to offload some of the LSFG processing to the unused tensor cores (if possible), to overall improve performance

This is already similar to how LSFG can choose between iGPU and dGPU in processing its load. Sometimes, using iGPU for LSFG can gain benefits. (offloading the processing to the iGPU instead of the discrete GPU)

Not everyone has an iGPU, and yet the option is still there. Not everyone has an HDR monitor, and yet, the option is there. Same goes for other features like VRR, etc. Case in point, An added option to utilize tensor cores for RTX users (if possible) would be nice legit.

[M]

isnt the motto of LS dev "frame generation for everybody"? I dont think, we should make a lossless scaling version, that can be used only by a few people or split the community. I think, its good as it is, dev gives his best to improve the app and supports it in general. thats a privilege for us thats not a matter of course.

btw, I love the lossless scaling app and I recommend it whenever applicable!

[Kachunnga]

Wondering if there's a platform-agnostic way of utilizing RT cores. My ONE major gripe with this program so-far is how computationally expensive frame gen can be at higher framerates/resolutions. Obviously it can't come free, just would be even more of a miracle program if it could use RT cores to both get better framerates and maybe even save power consumption.

(Still though love this program)

[Hakisak]

Originally posted by ogioto:

Already discussed previously- using these core won't make the visuals better due to the indirect way LS captures the frames. So, the tensor cores wouldn't be too much of a help in that case.
Also, the dev is against adding options that can be used only by specific users, which also adds against making such option.

Sorry I should searched more. Its a shame visuals can't be improved, but to be clear, I think in its current state its very very good, its just a shame it can't be taken up another step using hardware.

Originally posted by lyndonguitar:

I believe the aim of this suggestion isn't to "improve visuals", but to offload some of the LSFG processing to the unused tensor cores (if possible), to overall improve performance

This is already similar to how LSFG can choose between iGPU and dGPU in processing its load. Sometimes, using iGPU for LSFG can gain benefits. (offloading the processing to the iGPU instead of the discrete GPU)

Not everyone has an iGPU, and yet the option is still there. Not everyone has an HDR monitor, and yet, the option is there. Same goes for other features like VRR, etc. Case in point, An added option to utilize tensor cores for RTX users (if possible) would be nice legit.

Yes. the suggestion wasn't entierly about visuals, but about offloading onto unused parts of hardware. I just asked about visuals once. but the post is about offloading onto Tensor Cores.

I saw the option but didnt think about selecting an igpu to do the frame generation, I will have to try that to see what it's like!

I am %100 in line with you on that last part. well writen. rtx 2000 was released almost 6 years ago, its not fair for others to say rtx is for "specific users", the steam hardware survey shows that there are a large number of people using rtx cards.

Originally posted by M:

isnt the motto of LS dev "frame generation for everybody"? I dont think, we should make a lossless scaling version, that can be used only by a few people or split the community. I think, its good as it is, dev gives his best to improve the app and supports it in general. thats a privilege for us thats not a matter of course.

btw, I love the lossless scaling app and I recommend it whenever applicable! :Deacon_Binoculars:

again, LS has many features for many people, but why stop at rtx? rtx 2000 was released almost 6 years ago, in 4 years time you can say "rtx 2000 was released 10 years ago, why isn't this developer using Tensor Cores to improve this application?"

[Xavvy]

Originally posted by Hakisak:

Originally posted by ogioto:

Already discussed previously- using these core won't make the visuals better due to the indirect way LS captures the frames. So, the tensor cores wouldn't be too much of a help in that case.
Also, the dev is against adding options that can be used only by specific users, which also adds against making such option.

Sorry I should searched more. Its a shame visuals can't be improved, but to be clear, I think in its current state its very very good, its just a shame it can't be taken up another step using hardware.

Originally posted by lyndonguitar:

I believe the aim of this suggestion isn't to "improve visuals", but to offload some of the LSFG processing to the unused tensor cores (if possible), to overall improve performance

This is already similar to how LSFG can choose between iGPU and dGPU in processing its load. Sometimes, using iGPU for LSFG can gain benefits. (offloading the processing to the iGPU instead of the discrete GPU)

Not everyone has an iGPU, and yet the option is still there. Not everyone has an HDR monitor, and yet, the option is there. Same goes for other features like VRR, etc. Case in point, An added option to utilize tensor cores for RTX users (if possible) would be nice legit.

Yes. the suggestion wasn't entierly about visuals, but about offloading onto unused parts of hardware. I just asked about visuals once. but the post is about offloading onto Tensor Cores.

I saw the option but didnt think about selecting an igpu to do the frame generation, I will have to try that to see what it's like!

I am %100 in line with you on that last part. well writen. rtx 2000 was released almost 6 years ago, its not fair for others to say rtx is for "specific users", the steam hardware survey shows that there are a large number of people using rtx cards.

Originally posted by M:

isnt the motto of LS dev "frame generation for everybody"? I dont think, we should make a lossless scaling version, that can be used only by a few people or split the community. I think, its good as it is, dev gives his best to improve the app and supports it in general. thats a privilege for us thats not a matter of course.

btw, I love the lossless scaling app and I recommend it whenever applicable! :Deacon_Binoculars:

again, LS has many features for many people, but why stop at rtx? rtx 2000 was released almost 6 years ago, in 4 years time you can say "rtx 2000 was released 10 years ago, why isn't this developer using Tensor Cores to improve this application?"

I agree with everything you are saying and what the lyn was saying too. Also to add to this. It doesn't matter about tensor cores per say. People forget that AMD has Stream cores which are very similar so this can be co-opted for both types of hardware and I think this absolutely should be an option for the dev to consider if it's possible. If it's not possible, no harm done. But I think they should look into it. It's literally possible hardware resources for at least 50%+ of users not being utilised.

To more accurately sum this up. The only truly exclusive hardware that Nvidia uses is optical accelerator flow cores.. Tensor cores are used for AI upscaling such as DLSS, DLDSR and ray/path tracing and ray reconstruction and to be completely fair in the RTX 3000 and 4000 series GPUs only very rarely are even half of these cores even used. Case in point of a RTX 3080ti being able to run 8K DLDSR + DLSS Quality and RTX and still only using 80% of the cores. Nvidia put more of these cores than you will ever use into these GPUs and AMD puts an insane amount of stream processors into theirs.

[Jo2]

Tensors would be helpfull, no idea how it can be done but what i know that even 4090 struggling with generating extra 120fps to 240fps without performance losing for normal frames

[Reme86]

Originally posted by lyndonguitar:

Originally posted by ogioto:

Already discussed previously- using these core won't make the visuals better due to the indirect way LS captures the frames. So, the tensor cores wouldn't be too much of a help in that case.
Also, the dev is against adding options that can be used only by specific users, which also adds against making such option.

I believe the aim of this suggestion isn't to "improve visuals", but to offload some of the LSFG processing to the unused tensor cores (if possible), to overall improve performance

This is already similar to how LSFG can choose between iGPU and dGPU in processing its load. Sometimes, using iGPU for LSFG can gain benefits. (offloading the processing to the iGPU instead of the discrete GPU)

Not everyone has an iGPU, and yet the option is still there. Not everyone has an HDR monitor, and yet, the option is there. Same goes for other features like VRR, etc. Case in point, An added option to utilize tensor cores for RTX users (if possible) would be nice legit.

Especially relevant as its not just Nvidia that will be using hardware acceleration for this stuff going forwards. AMD and Intel will be going hardware rather than software based soon as well

[Gizzmoe]

Originally posted by - EMPTY -:

Its impossible to generate interpolated frame at zero cost. Its like any graphical settings in any games, you add more shadow you loose frames, you add more interpolated frame you loose frame etc.

That depends on the situation and the viewpoint. For example, you have a 60hz monitor and a decent GPU, the GPU is bored and uses only 60% power in games at 60hz, then you upgrade to 240hz and can suddenly do a 60x3/4 framegen at "zero cost" (just higher GPU usage), because the GPU has enough headroom.

[SpeedFreak3]

Using the tensor cores of of say RTX 20 or 30 series for LSFrameGen while Nvidia locks them out of their official FrameGen would be kind of ironic, i love that idea.

[Gemini]

I also believe that this part of the core should be fully utilized, exchanging idle core parts for performance improvement

[8bit shadow]

you guys do realise that both AMD and Nvidia cards, intel cards too iirc, have tensor cores right?

this wouldn't limit it to anyone, just 8+ year old cards at best - and would be a toggle if applicable.

or do we just get rid of 4x and higher frame gen because 'not everyone can use it'?
That's clearly stupid.

There's a difference between locking a feature on software for some artificial BS reason and locking it because the hardware literally doesn't support it.

[ogioto]

Originally posted by Reme86:

Originally posted by lyndonguitar:

I believe the aim of this suggestion isn't to "improve visuals", but to offload some of the LSFG processing to the unused tensor cores (if possible), to overall improve performance

This is already similar to how LSFG can choose between iGPU and dGPU in processing its load. Sometimes, using iGPU for LSFG can gain benefits. (offloading the processing to the iGPU instead of the discrete GPU)

Not everyone has an iGPU, and yet the option is still there. Not everyone has an HDR monitor, and yet, the option is there. Same goes for other features like VRR, etc. Case in point, An added option to utilize tensor cores for RTX users (if possible) would be nice legit.

Especially relevant as its not just Nvidia that will be using hardware acceleration for this stuff going forwards. AMD and Intel will be going hardware rather than software based soon as well

Intel have been using such since they launched XeSS, on their GPUs it runs better because it uses their XMX cores.

Originally posted by 8bit shadow:

you guys do realise that both AMD and Nvidia cards, intel cards too iirc, have tensor cores right?

this wouldn't limit it to anyone, just 8+ year old cards at best - and would be a toggle if applicable.

or do we just get rid of 4x and higher frame gen because 'not everyone can use it'?
That's clearly stupid.

There's a difference between locking a feature on software for some artificial BS reason and locking it because the hardware literally doesn't support it.

The main issue is that adapting the tech to the different type of tensor cores will be hell that would eat too much time. The nvidia tensor cores and the intel XMX cores work in different ways, as will the new AMD RDNA tensor cores. So, that would require way too much adapting on too many fronts for a singular dev.

PS: At the time of the original answer, at the start of the thread, not all other companies had tensor cores in their standard customer products