yeah OVA there is some issues with CBR this has been since release of the Media SDK from AMD. it seems to hard encorce the max bitrate for the resolution somehow.It feels like it just ignores CBR flag. I just pressed "Use custom parameters" And pressed "Quality" button. Also AMF crashes video card on both Omega and Beta drivers, so i use OVA.
so have you tried changing some settings with AMF or does it crash as soon as you hit the button?