Bug Report Replay buffer problems with NVENC [Fix found]

[pollyzoid]

First bug is that the replay buffer doesn't seem to limit its length according to what I've set in the settings.

Even with a length of 300 seconds (5 minutes), OBS kept a buffer way longer than that. The first buffer was 1h 44min long, second try about 8 minutes. Trying to stop the 1h buffer resulted in OBS using over 4GB of memory and not responding in over a minute (force closed after that), no file saved. 8 minute file saved successfully.

Logs:
1h 44min buffer and force close
8min buffer and save

Second bug is that the 'Save Replay Buffer' hotkey doesn't work. Every other hotkey works, even with the same key set. I've used 'Record from Replay Buffer' and then 'Stop Recording' to get the same effect, but it would be nice if the actual hotkey worked.
Trying to use 'Save' results in identical logs as not using it.

Windows 8.1, 64-bit
Open Broadcaster Software v0.638b - 64bit
Gigabyte GTX 970

 

[dodgepong]

Keyframe interval of -1? Did you set that yourself? That seems unusual.

 

[pollyzoid]

Nope, it's set to 0 in options.

On further testing, both bugs seem to be related to using NVENC. Using x264 (with buffer length 30) correctly limits the saved buffer length to about 30 seconds (or about 35 in most of my tests).
The hotkey is also fixed by using x264, but I also noticed that ending the replay buffer doesn't save it to disk by itself, like it did with NVENC. So maybe NVENC is limited to only save when ending the buffer, while x264 is capable of doing that mid-buffer?

Settings I used for the 1h 44min buffer:
Encoding:

Advanced:

 

[cvax]

Nope, it's set to 0 in options.

On further testing, both bugs seem to be related to using NVENC. Using x264 (with buffer length 30) correctly limits the saved buffer length to about 30 seconds (or about 35 in most of my tests).
The hotkey is also fixed by using x264, but I also noticed that ending the replay buffer doesn't save it to disk by itself, like it did with NVENC. So maybe NVENC is limited to only save when ending the buffer, while x264 is capable of doing that mid-buffer?

Settings I used for the 1h 44min buffer:
Encoding:

Advanced:

Can confirm this bug seems to be related to NVENC. Saving buffer hot key does not work when encoded with NVENC.

Fix seems to be manually changing keyframe from 0 to 1.

 

[true-meaning]

hah,same problem with me,i find the answer in the source code,maybe i should push a request in github
https://obsproject.com/forum/threads/replaybuffer-out-of-whack-when-use-nvenc-encoder.24363/

 

[pollyzoid]

I can confirm, setting Keyframe Interval to 1 in Advanced settings fixes both of the bugs I listed in the OP. Makes sense in retrospect.
Thanks to both of the above posters for figuring it out.