Comparison of x264, NVENC, Quicksync, VCE

koala · Nov 9, 2016

@Xaymar The color range is still full, not partial. Intended? MediaInfo says this, and ffprobe as well. This consumes a bit of bandwidth.

Media Player Classic and the "Movie & TV" app from Windows 10 cannot display the colors from your videos correctly. They both black out the darker parts, which means they process the footage as partial, while it is in fact full color range. Ffmpeg, on the other hand, correctly identifies and processes as full range, which you can see by the fact that if you export a frame as png with ffmpeg, the png has correct colors. The same frame displayed with the media players give wrong colors.
Your videos are definitely full range, otherwise the media players would not make the darker parts too dark. If your videos were partial and the media players process them as full, the darker parts would be too light instead of too dark.

The comparison and rating of my script is correct regardless the full range, because ffmpeg correctly takes the color range in account.

Xaymar · Nov 9, 2016

koala said:
@Xaymar The color range is still full, not partial. Intended? MediaInfo says this, and ffprobe as well. This consumes a bit of bandwidth.

...

The comparison and rating of my script is correct regardless the full range, because ffmpeg correctly takes the color range in account.

MPC-HC, VLC and ffmpeg should have no problems with these files, only those that incorrectly read the h264 stream have issues. I can do them over with partial range if needed, but I'd like to see where these rank first.

Edit: Yes, Full Range is intended. It finally works this time as i fixed it in the plugin itself.

Suslik V · Nov 9, 2016

I can confirm that qsv, nvidia and x264 videos are partial range, as I see.

koala · Nov 9, 2016

Ok, here is the automated comparison including all 3 hardware encoders again. I left every quality setting of the other encoders instead of removing the inferior ones, so you can see what the difference is. AMD-vce is not yet at the quality of the other two hardware encoders.

Visual comparison: if you visually compare the videos near the low-motion reference frame, that is after the hero harvested the plant at the beginning of the video, you definitely see less detail of the snow around and behind him with vce than with all other encoders. And while he's running to the tent, you see the fog of compression artefacts around him with vce, while this fog is much less visible with the others.

1280x720 30 fps:

stream encoder comparison 1280x720 30fps +amd.png

raw tab-delimited source file

1280x720 60 fps:

stream encoder comparison 1280x720 60fps +amd.png

raw tab-delimited source file

1920x1080 30 fps:

stream encoder comparison 1920x1080 30fps +amd.png

raw tab-delimited source file

Suslik V · Nov 9, 2016

@Xaymar , may it happen that (height>=780) was a typo from AMD staff and actual value is >=720p?
_______________________________________________________________

koala said:
...regardless the full range...

I always thought, that if information was lost - you cannot restore it.

Xaymar · Nov 9, 2016

Suslik V said:
@Xaymar , may it happen that (height>=780) was a typo from AMD staff and actual value is >=720p?
_______________________________________________________________

I always thought, that if information was lost - you cannot restore it.

No, it actually didn't work. It would've been too CPU-intensive too, since it would do the coversion on CPU if I understood Jim in IRC correctly.

Xaymar · Nov 9, 2016

koala · Nov 10, 2016

For that kind of doing the same recordings over and over again, I created the script ;)
Here is the rating list of your new videos, where the vce recordings appear.

stream encoder comparison 1280x720 30fps +amd 2.png

raw tab-delimited source file

Suslik V · Nov 10, 2016

OK. I replaced some AMD files from my tests, 1280x720@30 2000kbit/s, new list:

simple - stream - x264 bitrate=2000 preset=veryfast.mp4
simple - stream - NVENC_Pascal bitrate=2000 preset=highquality.mp4
simple - stream - NVENC_Kepler bitrate=2000 preset=highquality.mp4
simple - stream - QSV_Skylake bitrate=2000 preset=balanced.mp4
simple - stream - QSV_IvyBridge bitrate=2000 preset=balanced.mp4
simple - stream - QSV_Skylake bitrate=2000 preset=quality.mp4
simple - stream - QSV_Skylake bitrate=2000 preset=speed.mp4
VCE1415_1280x720x30_2000_Balanced_Partial.mkv
VCE1415_1280x720x30_2000_Balanced_Partial_NoBPictures.mkv

Now the rating is (almost the same as previous):

NVENC_Pascal (2) best
NVENC_Kepler & x264 (3 & 1)
QSV_IvyBridge (4)
QSV_Skylake (6 & 5)
AMD_VCE1415_NoBPictures (9)
AMD_VCE1415 (8)

All tests results is by my own perception of the footage.
All previous comments are still true. Want to say a few words about AMD encoder (as it arrived last and from the second try).

First, AMD_VCE1415_NoBPictures (9) and AMD_VCE1415 (8) files - results are much better, than first examples I saw (thanks to @Xaymar !!! Well done. Respect to all who helped and points you.). Less blocking, same range and color space. Result close to simple - stream - QSV_Skylake bitrate=2000 preset=speed.mp4 (7) - I didn't place it in the chart table because Quality preset is better for Intel as it was mentioned by me earlier. But temporal artifacts (there I mean visible keyframes insertion - whole image just replaced with same picture but different quality) are make AMD videos out of the list in comparison to other hw based encoders at 1280x720@30 2000.

Second, Quality presets for AMD I excluded because of this dammit temporal degradation. When overall quality is better, this bad keyframes insertion completed in timeline makes video even worse than it was for Speed preset (maybe keyframes size or time are bad?). Speed preset has lower overall quality and thus this keyframes appearance are smoother, but still bad (well noticeable at timeline from 00:01:01 to 00:01:18, when hero "landing"). So, Speed preset excluded from the test.

Third, AMD and B-frames. As I see, there is two versions of recorded footage with B-Pictures (VCE2, VCE3) and without B-Pictures (VCE1, VCE3.4) - last marked as NoBPictures. And this videos differ in quality. And differ a lot!
All videos marked as NoBPictures are better to me than any with B-pictures. NoB-fames videos are less noisy and crisper. That's why I placed NoBPictures videos higher.
Again, Speed preset with B-frames has noticeable flickering at background (see timeline from 00:00:48 to 00:01:17, flying hero - clouds left to him, dark side of the portal-tower and ground where hero is landing) thus file excluded from the test.

Fourth, Balanced preset for AMD was chosen mainly because most "smooth" temporal quality degradation (there I mean visible keyframes insertion). It is still visible but I think it less noticeable. I may wrong in this part, but Quality preset make it visible even crisper (this is bad). Look at timeline from 00:00:32 to 00:00:38 after camera moves and hero "landing" - from 00:01:01 to 00:01:18 - whole image "stutter" when new keyframe inserted, all videos affected.

Fifth, Few words to AMD staff. They don't hear this because this is different place ^_^ but now we can compare their engineer's work. Because, earlier it was invisible what they are working on and how good it is. Results are not so good... At least it works, but any other do it better. Height>=780 equal to HD by AMD is weird for me. I thought, 720p is HD. I thought, that more expensive cards are better in all (and they don't).

Conclusion.
NVIDIA Pascal is a king of low bitrate streaming *_*
That it. Nothing to add. Hope, things can change in the future (still, this is too expensive card).

BT.2020, UHDTV, 10-12bit, HEVC, is far tomorrow. You are limited to use things available right now.

chummy · Nov 10, 2016

B-frames have bad quality in high motion for all hardware encoders Quicksync, Nvenc too. Using many of them like OBS does for Quicksync(7bframes) without b-pyramid cause too much blocky artifacting. My Haswell Quicksync when using 0 b-frames in high motion give better quality. B-pyramid is a must to smooth out frames and avoid too much artifacting/blockiness. OBS-Studio using 7 b-frames is too much and without b-pyramid for Quicksync is the biggest culprit of worse quality.

sam686 · Nov 10, 2016

Looking at the video through MediaInfo, there are a few differences.

Format Profile: x264 use High profile, but not for UltraFast. AMD VCE also use high profile. If you can get NVENC and QuickSync to use high profile, then they may compress better or improve compression quality. The only reason for limiting to main profile is for older mobile devices mostly for those that are limited to 720p 30fps will only support main or baseline profile.

Matrix Coefficients: NVENC have it set to BT.601, but others just have them undefined or not shown in Mediainfo. This may cause to wrong colors for ffmpeg as they default to bt.709 for 720p and higher when letting ffmpeg convert and output to RGB. ffmpeg allows overriding the decoder color matrix in video files. For x264, a custom option colormatrix=bt470bg or bt470m could be used, but not sure which one to use.

B-Frames: I checked the videos through ffmpeg visualizations. x264 use dynamic number of b-frames up to 3 in the default VeryFast and Medium, which can improve quality. Quicksync use 7 b-frames between p-frame and no dynamic b-frames? No wonder quality is not as good as to me it looks like video have a small motion flicker every 7 or 8 frames, especially the slow moving clouds in the sky at 35 seconds in the video. ~~NVENC use 3, and AMD VCE use 4.~~ None of the hardware encoder have dynamic b-frames. I would like to see QuickSync with 1 or 0 b-frames. EDIT: I was wrong on the number of bframes for both NVENC and AMD VCE.

koala · Nov 10, 2016

I used the simple settings of OBS for the analysis, and here you cannot define anything else than the preset. You can give a user-defined option string, but I don't know what options the hardware encoders support. I assume, OBS defaults to main profile for maximum stream compatibility for devices. Perhaps streaming services also require main mode. Beyond that, Tests I conducted for quality-based rate control modes showed that a quality difference between main profile and high profile exist for some videos, but is very small. You will probably not notice it (much).

You can switch to advanced mode and define some more options. For Quicksync, there is only an "Async Depth" option available, from which I don't know what this does. Probably some kind of lookahead or parallel-processing of frames within the encoder.

If you give me explicit settings definitions, or a range of settings to try, I will happily produce as much test videos with that, advanced mode, if you like. But my own knowledge ends here. I tend to not wander too far away from the defaults. I prefer using settings that give 90% performance and runs on 90% of the machines instead of settings that give 100% performance but only work on 10% of the machines.

Xaymar · Nov 10, 2016

sam686 said:
,,, AMD VCE use 4.

AMD VCE should be using 3, at least according to the log file and debug output.

sam686 · Nov 11, 2016

Xaymar said:
AMD VCE should be using 3, at least according to the log file and debug output.

My bad, its hard to see the visualizations until I zoom in. It does look like NVENC uses 2 b-frames, or 1 less then AMD VCE.

Another problem was seeking was kindof slow in your MKV video files in both VLC and media player classic with ffdshow. This slow seek problem is gone after I pass it through ffmpeg to re-mux using one of the following command:

Code:

ffmpeg -i "VCE1415_1280x720x30_3500_Quality_Partial.mkv" -codec copy -movflags faststart "VCE1415_1280x720x30_3500_Quality_Partial_remux.mp4
ffmpeg -i "VCE1415_1280x720x30_3500_Quality_Partial.mkv" -codec copy "VCE1415_1280x720x30_3500_Quality_Partial_remux.mkv"

Xaymar · Nov 11, 2016

Seeking issue is known, but there is not much I can do about it. It plays fine in MPC-HC and the timestamps are also correctly set from my end. DTS (Decode TimeStamp) is in the same order as the encoder outputs, PTS (Presentation TimeStamp) is in order that the frames come in. I don't know why VLC, WMP and some other players have so much trouble reading the timestamps properly.

koala · Nov 11, 2016

I added a documentation and the test script itself here:
test script
documentation
Should be possible to make it work on other machines than my own. It would be much easier, if OBS were able to provide a scriptable interface so you can automate interaction with the GUI, but currently there isn't anything like this.

Suslik V · Nov 12, 2016

Just want to add few words, about my viewing experience of AMD VCE videos.
I googled a bit and found term "keyframe pumping", usually it mentioned when quality has degradation in time and improves when IDR inserted. But sometimes it goes vise versa - this what I see in the AMD VCE videos. New IDR frame has lower quality than previous image sequence (maybe few macroblocks before it encoded at higher quality). IDR frame size (quality) probably restricted by max bitrate or so.

koala · Nov 13, 2016

Table of bitrate/resolution/fps matrix for easy visual video comparison:

- Color Format: NV12 (OBS default)
- YUV Color Space: 601 (OBS default)
- YUV Color Range: Partial (OBS default)

x264 "veryfast"
--- 1280x720 1280x720 1920x1080 1920x1080
2000 30 fps | 60 fps | 30 fps | 60 fps
2500 30 fps | 60 fps | 30 fps | 60 fps
3500 30 fps | 60 fps | 30 fps | 60 fps
5000 30 fps | 60 fps | 30 fps | 60 fps
6000 30 fps | 60 fps | 30 fps | 60 fps

nvenc "high quality"
--- 1280x720 1280x720 1920x1080 1920x1080
2000 30 fps | 60 fps | 30 fps | 60 fps
2500 30 fps | 60 fps | 30 fps | 60 fps
3500 30 fps | 60 fps | 30 fps | 60 fps
5000 30 fps | 60 fps | 30 fps | 60 fps
6000 30 fps | 60 fps | 30 fps | 60 fps

quicksync "quality"
--- 1280x720 1280x720 1920x1080 1920x1080
2000 30 fps | 60 fps | 30 fps | 60 fps
2500 30 fps | 60 fps | 30 fps | 60 fps
3500 30 fps | 60 fps | 30 fps | 60 fps
5000 30 fps | 60 fps | 30 fps | 60 fps
6000 30 fps | 60 fps | 30 fps | 60 fps

amd vce "high quality"
Remark: all vce videos have YUV Color Space: 706, and the 1280x720x60 and 1920x1080x30 have YUV Color Range: full.
--- 1280x720 1280x720 1920x1080 1920x1080
2000 30 fps | 60 fps | 30 fps | 60 fps
2500 30 fps | 60 fps | 30 fps | 60 fps
3500 30 fps | 60 fps | 30 fps | 60 fps

Faleene · Nov 20, 2016

Suslik V said:
Just want to add few words, about my viewing experience of AMD VCE videos.
I googled a bit and found term "keyframe pumping", usually it mentioned when quality has degradation in time and improves when IDR inserted. But sometimes it goes vise versa - this what I see in the AMD VCE videos. New IDR frame has lower quality than previous image sequence (maybe few macroblocks before it encoded at higher quality). IDR frame size (quality) probably restricted by max bitrate or so.

I've noticed this too. Especially visible at lower bitrate. The feed basically pulses because the first IDR frame is just awful quality for whatever reason

koala · May 17, 2017

Since I still find a continuous flow of downloads of the demo videos for comparison, I added 5000 and 6000 bitrate videos to the above comparison table.

According to Twitch, they now allow streams up to 6000, so this might be of some interest.

Comparison of x264, NVENC, Quicksync, VCE

Active Member

Active Member

Active Member

Active Member

Active Member

Active Member

Active Member

Active Member

Active Member

Member

Member

Active Member

Active Member

Member

Active Member

Active Member

Active Member

Active Member

New Member

Active Member