I'd definitely stick to AAC audio.
Its gotta be something else if you have CFR enabled. What bitrate or quality setting are you using? I've seen things like this happen when people attempt a constant rate factor that's too low, like CRF=0 or CRF=1.
Post a log file and make a small sample .mp4 (maybe one minute long) that you verify crashes Vegas and upload it to a file sharing site so we can download it and see for ourselves.
Okay, I changed it back to AAC. I have "Custom x264 Encoder Settings" checked and (without quotes) "crf=15" in the box. Here's the sample clip, I figured Mediafire will do the job:
https://www.mediafire.com/?0u2uw4hbic2g6nx
Below is the log:
00:01:59: =====Stream Start: 2015-01-13, 00:01:59===============================================
00:01:59: Multithreaded optimizations: On
00:01:59: Base resolution: 1366x768
00:01:59: Output resolution: 1364x768
00:01:59: ------------------------------------------
00:01:59: Loading up D3D10 on NVIDIA GeForce GT 635M (Adapter 1)...
00:01:59: ------------------------------------------
00:01:59: Audio Format: 48000 Hz
00:01:59: ------------------------------------------
00:01:59: Audio Channels: 2 Ch
00:01:59: Playback device {0.0.0.00000000}.{753b5d74-3450-498b-ab63-2cc82e319ecd}
00:01:59: ------------------------------------------
00:01:59: Using desktop audio input: Speakers (Realtek High Definition Audio)
00:01:59: Global Audio time adjust: 0
00:01:59: ------------------------------------------
00:01:59: Using auxilary audio input: Microphone (4- AT2020 USB )
00:01:59: Mic time offset: 0
00:01:59: ------------------------------------------
00:01:59: Audio Encoding: AAC
00:01:59: bitrate: 192
00:01:59: Using graphics capture
00:01:59: Trying to hook process: javaw.exe
00:01:59: Scene buffering time set to 700
00:01:59: Using custom x264 settings: "crf=15"
00:01:59: x264: VBV maxrate specified, but no bufsize, ignored
00:01:59: ------------------------------------------
00:01:59: Video Encoding: x264
00:01:59: fps: 60
00:01:59: width: 1364, height: 768
00:01:59: preset: ultrafast
00:01:59: profile: high
00:01:59: keyint: 250
00:01:59: CBR: no
00:01:59: CFR: yes
00:01:59: max bitrate: 1000
00:01:59: buffer size: 0
00:01:59: quality: 17
00:01:59: ------------------------------------------
00:01:59: using memory capture
00:03:42: FlushBufferedVideo: Flushing 34 packets over 550 ms
00:03:43: Total frames encoded: 6175, total frames duplicated: 16 (0.26%)
00:03:43: Total frames rendered: 6190, number of late frames: 2 (0.03%) (it's okay for some frames to be late)
00:03:43:
00:03:43: Profiler time results:
00:03:43:
00:03:43: ==============================================================
00:03:43: video thread frame - [100%] [avg time: 1.036 ms] [children: 13.9%] [unaccounted: 86.1%]
00:03:43: | scene->Preprocess - [0.0965%] [avg time: 0.001 ms]
00:03:43: | GPU download and conversion - [13.8%] [avg time: 0.143 ms] [children: 10%] [unaccounted: 3.76%]
00:03:43: | | flush - [7.63%] [avg time: 0.079 ms]
00:03:43: | | CopyResource - [2.22%] [avg time: 0.023 ms]
00:03:43: | | conversion to 4:2:0 - [0.193%] [avg time: 0.002 ms]
00:03:43: Convert444Threads - [100%] [avg time: 0.504 ms] [children: 99.2%] [unaccounted: 0.794%]
00:03:43: | Convert444toNV12 - [99.2%] [avg time: 0.5 ms]
00:03:43: encoder thread frame - [100%] [avg time: 0.845 ms] [children: 13%] [unaccounted: 87%]
00:03:43: | sending stuff out - [13%] [avg time: 0.11 ms]
00:03:43: ==============================================================
00:03:43:
00:03:43:
00:03:43: Profiler CPU results:
00:03:43:
00:03:43: ==============================================================
00:03:43: video thread frame - [cpu time: avg 0.451 ms, total 2796.88 ms] [avg calls per frame: 1]
00:03:43: | scene->Preprocess - [cpu time: avg 0 ms, total 0 ms] [avg calls per frame: 1]
00:03:43: | GPU download and conversion - [cpu time: avg 0.131 ms, total 812.5 ms] [avg calls per frame: 1]
00:03:43: | | flush - [cpu time: avg 0.093 ms, total 578.125 ms] [avg calls per frame: 1]
00:03:43: | | CopyResource - [cpu time: avg 0.015 ms, total 93.75 ms] [avg calls per frame: 1]
00:03:43: | | conversion to 4:2:0 - [cpu time: avg 0.007 ms, total 46.875 ms] [avg calls per frame: 1]
00:03:43: Convert444Threads - [cpu time: avg 0.651 ms, total 8000 ms] [avg calls per frame: 2]
00:03:43: | Convert444toNV12 - [cpu time: avg 0.651 ms, total 8000 ms] [avg calls per frame: 2]
00:03:43: encoder thread frame - [cpu time: avg 0.211 ms, total 1296.88 ms] [avg calls per frame: 1]
00:03:43: | sending stuff out - [cpu time: avg 0.078 ms, total 484.375 ms] [avg calls per frame: 1]
00:03:43: ==============================================================
00:03:43:
00:03:43: =====Stream End: 2015-01-13, 00:03:43=================================================
00:03:51: Terminating 0x8160
00:03:51: Terminating 0x8ae4
00:03:51: Terminating 0x7f28
Last game capture log:
2015-01-13, 00:01:12: we're booting up:
00:01:12: CaptureThread: attached to process javaw.exe
00:01:12: GL Present
00:01:12: DirectDraw capture: Windows 8 not supported yet
00:01:12: (half life scientist) everything.. seems to be in order
00:01:19: SwapBuffers(3405846223) Called
00:01:19: NV Capture available
00:01:19: FBO available
00:01:19: setting up gl data
00:01:19: DoGLGPUHook: wglDXOpenDeviceNV failed
00:01:19: ---------------------- Cleared OpenGL Capture ----------------------
00:01:19: DoGLCPUHook: success
00:01:19: wglSwapBuffers(3405846223) Called
00:01:19: setting up gl data
00:01:19: SwapBuffers(3405846223) Called
00:01:19: successfully capturing gl frames via RAM
00:01:20: DirectDraw capture: Windows 8 not supported yet
00:01:37: stop requested, terminating gl capture
00:01:37: ---------------------- Cleared OpenGL Capture ----------------------
00:01:37: DoGLGPUHook: wglDXOpenDeviceNV failed
00:01:37: ---------------------- Cleared OpenGL Capture ----------------------
00:01:37: DoGLCPUHook: success
00:01:37: wglSwapBuffers(3405846223) Called
00:01:37: setting up gl data
00:01:37: SwapBuffers(3405846223) Called
00:01:40: DirectDraw capture: Windows 8 not supported yet
00:01:53: successfully capturing gl frames via RAM
00:01:58: stop requested, terminating gl capture
00:01:58: ---------------------- Cleared OpenGL Capture ----------------------
00:01:58: DoGLGPUHook: wglDXOpenDeviceNV failed
00:01:58: ---------------------- Cleared OpenGL Capture ----------------------
00:01:58: DoGLCPUHook: success
00:01:58: SwapBuffers(3405846223) Called
00:01:58: setting up gl data
00:01:58: wglSwapBuffers(3405846223) Called
00:01:59: successfully capturing gl frames via RAM
00:02:00: DirectDraw capture: Windows 8 not supported yet
00:03:43: stop requested, terminating gl capture
00:03:43: ---------------------- Cleared OpenGL Capture ----------------------
00:03:43: DoGLGPUHook: wglDXOpenDeviceNV failed
00:03:43: ---------------------- Cleared OpenGL Capture ----------------------
00:03:43: DoGLCPUHook: success
00:03:43: wglSwapBuffers(3405846223) Called
00:03:43: setting up gl data
00:03:43: SwapBuffers(3405846223) Called
00:03:46: DirectDraw capture: Windows 8 not supported yet
00:03:47: wglDeleteContext Called
00:03:47: ---------------------- Cleared OpenGL Capture ----------------------