Bug Report No option to use NVENC(Old) in the new update.


New Member
Hello! I decided to create an account specifically to add my findings to this topic, as I also find there are differences between the two codecs.

I am using the GTX 1060 Pascal card.
Judging by the discussions, the parameters passed to the card/codec make it behave differently, and I will post below my findings. Please consider them and include the "old" NVENC codec / API as well.

1) I recorded some Quake 3 gameplay and then replayed the video in fullscreen, recording it with OBS - Display capture. These were the results for the same 35 second clip, using CQP mode - comparing the bitrate of the resulted video streams:

For the old Nvenc CQP settings, I usually settled to CQ 23 at 1080p since it produced quite acceptable quality for local recordings.

So these are the results between using the old vs new NVENC in OBS 24.0.3 64-bit - notice the bitrate differences while only changing CQ parameter and keeping all of the other settings identical :
Nvenc old CQ 23 = 24116 kbps

Nvenc new CQ 23 = 31888 kbps
Nvenc new CQ 24 = 28930 kbps
Nvenc new CQ 25 = 26147 kbps
Nvenc new CQ 25 Psycho Visual OFF = 27580 kbps - so worse, keep that ON
Nvenc new CQ 26 = 23329 kbps

Now for quality: no discernable quality differences between the two - so can use Nvenc New with CQ 26 instead of Nvenc Old CQ 23!
2) I decided to take the same test on some Doom 2016 game footage I had laying around, and here as well, I needed to set the new encode to CQ 26 to match the bitrate and also the perceived quality of the old one:

Nvenc Old CQ 23 = 34044 kbps
Nvenc New CQ 26 = 34280 kbps

Quality? Seems like Nvenc New keeps more detail in some situations!

3) I had some older test laying around in a txt, of some GTA V gameplay, and here, while recording the benchmark (to have the same reference video footage), I noted the following:
These are the encoder settings when using the "Simple" NVENC encoder (using the "NVENC new" encoder: jim-nvenc)

20:03:47.242: [jim-nvenc: 'simple_h264_recording'] settings:
20:03:47.242:     rate_control: CQP
20:03:47.242:     bitrate:      0
20:03:47.242:     cqp:          23
20:03:47.242:     keyint:       250
20:03:47.242:     preset:       hq
20:03:47.242:     profile:      high
20:03:47.242:     width:        1920
20:03:47.242:     height:       1080
20:03:47.242:     2-pass:       false
20:03:47.242:     b-frames:     2
20:03:47.242:     lookahead:    false
20:03:47.242:     psycho_aq:    true

The "old" Nvenc encoder - with advanced setup will display "NVENC encoder" as follows:

20:26:37.447: [NVENC encoder: 'recording_h264'] settings:
20:26:37.447:     rate_control: CQP
20:26:37.447:     bitrate:      0
20:26:37.447:     cqp:          23
20:26:37.447:     keyint:       250
20:26:37.447:     preset:       hq
20:26:37.447:     profile:      high
20:26:37.447:     width:        1920
20:26:37.447:     height:       1080
20:26:37.447:     2-pass:       false
20:26:37.447:     b-frames:     2
20:26:37.447:     GPU:          0

Stats of recording pass 4 of GTA V benchmark with the two of them:

NVENC new (jim-nvenc), 2 minutes 8 seconds:
813 MB

NVENC old (NVENC encoder), 2 minutes 4 seconds:
605 MB

Quality differences? Very little - undiscernable!

  • Bitrate rate control shouldn't be affected as it seems quality is quite similar for similar bitrate, however
  • If using a CQP rate control, then the "new" encoder requires a higher value for the CQ to match the bitrate and quality of the older one.