Really need a log file to see more detail in regards to settings of OBS, etc.
Can you do a few minutes' worth of recording (Doing what you would normally do in-game) then stop the recording session, then upload a current logfile please so I or someone else can further assist in troubleshooting:
https://obsproject.com/forum/threads/please-post-a-log-with-your-issue-heres-how.23074/
Limiting your frame rate in-game will help a lot in getting the best quality out of your hardware in regards to recording/streaming. I recommend using MSI AfterBurner or EVGA Precision to cap your fps to your monitor refresh rate, or 1-3 frames higher than your monitor refresh rate will generally resolve the issue of screen tearing. Try to avoid V-Sync as it can induce input lag.
07:01:07.605: CPU Name: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
07:01:07.605: CPU Speed: 3392MHz
07:01:07.605: Physical Cores: 4, Logical Cores: 8
07:01:07.605: Physical Memory: 16291MB Total, 10426MB Free
07:01:07.605: Windows Version: 10.0 Build 16299 (revision: 248; 64-bit)
07:01:07.605: Running as administrator: false
07:01:07.605: Aero is Enabled (Aero is always on for windows 8 and above)
07:01:07.605: Windows 10 Gaming Features:
07:01:07.605: Game DVR: On
07:01:07.606: Sec. Software Status:
07:01:07.607: Windows Defender Antivirus: enabled (AV)
07:01:07.607: Windows Firewall: enabled (FW)
07:01:07.608: Windows Defender Antivirus: enabled (ASW)
07:01:07.608: Portable mode: false
07:01:07.625: OBS 21.0.1 (64bit, windows)
07:01:07.625: ---------------------------------
07:01:07.625: ---------------------------------
07:01:07.625: audio settings reset:
07:01:07.625: samples per sec: 48000
07:01:07.625: speakers: 2
07:01:07.626: ---------------------------------
07:01:07.626: Initializing D3D11...
07:01:07.626: Available Video Adapters:
07:01:07.628: Adapter 1: NVIDIA GeForce GT 640
07:01:07.628: Dedicated VRAM: 4261216256
07:01:07.628: Shared VRAM: 4246278144
07:01:07.628: output 1: pos={0, 0}, size={1920, 1080}, attached=true
07:01:07.628: output 2: pos={1920, 9}, size={1920, 1080}, attached=true
07:01:07.629: Loading up D3D11 on adapter NVIDIA GeForce GT 640 (0)
07:01:07.995: D3D11 loaded successfully, feature level used: 45056
07:01:08.641: ---------------------------------
07:01:08.641: video settings reset:
07:01:08.641: base resolution: 1920x1080
07:01:08.641: output resolution: 1920x1080
07:01:08.641: downscale filter: Lanczos
07:01:08.641: fps: 60/1
07:01:08.641: format: NV12
07:01:08.643: Audio monitoring device:
07:01:08.643: name: Default
07:01:08.643: id: default
07:01:08.644: ---------------------------------
07:01:08.645: Required module function 'obs_module_load' in module '../../obs-plugins/64bit/chrome_elf.dll' not found, loading of module failed
07:01:08.648: [CoreAudio encoder]: CoreAudio AAC encoder not installed on the system or couldn't be loaded
07:01:08.703: [AMF] <obs_module_load> AMF not supported (no AMD graphics)
07:01:08.703: Failed to initialize module 'enc-amf.dll'
07:01:08.721: Required module function 'obs_module_load' in module '../../obs-plugins/64bit/libcef.dll' not found, loading of module failed
07:01:08.722: Required module function 'obs_module_load' in module '../../obs-plugins/64bit/libEGL.dll' not found, loading of module failed
07:01:08.722: Required module function 'obs_module_load' in module '../../obs-plugins/64bit/libGLESv2.dll' not found, loading of module failed
07:01:08.723: Failed to load 'en-US' text for module: 'obs-browser.dll'
07:01:08.723: [browser_source: 'Version: 1.29.0']
07:01:08.768: NVENC supported
07:01:09.100: Couldn't find VLC installation, VLC video source disabled
07:01:09.110: No blackmagic support
07:01:09.217: ---------------------------------
07:01:09.217: Loaded Modules:
07:01:09.217: win-wasapi.dll
07:01:09.217: win-mf.dll
07:01:09.217: win-dshow.dll
07:01:09.218: win-decklink.dll
07:01:09.218: win-capture.dll
07:01:09.218: vlc-video.dll
07:01:09.218: text-freetype2.dll
07:01:09.218: rtmp-services.dll
07:01:09.218: obs-x264.dll
07:01:09.218: obs-vst.dll
07:01:09.218: obs-transitions.dll
07:01:09.218: obs-text.dll
07:01:09.218: obs-qsv11.dll
07:01:09.218: obs-outputs.dll
07:01:09.218: obs-filters.dll
07:01:09.218: obs-ffmpeg.dll
07:01:09.218: obs-browser.dll
07:01:09.218: image-source.dll
07:01:09.218: frontend-tools.dll
07:01:09.218: enc-amf.dll
07:01:09.218: coreaudio-encoder.dll
07:01:09.218: ---------------------------------
07:01:09.218: ==== Startup complete ===============================================
07:01:09.235: All scene data cleared
07:01:09.236: ------------------------------------------------
07:01:09.274: WASAPI: Device 'Headset Earphone (2- ASTRO MixAmp Pro )' initialized
07:01:09.295: WASAPI: Device 'Microphone (Yeti Stereo Microphone)' initialized
07:01:09.344: adding 21 milliseconds of audio buffering, total audio buffering is now 21 milliseconds
07:01:09.512: adding 21 milliseconds of audio buffering, total audio buffering is now 42 milliseconds
07:01:09.952: [window-capture: 'Osrs'] update settings:
07:01:09.952: executable: OSBuddy64.exe
07:01:09.954: Switched to scene 'Minecraft'
07:01:09.955: ------------------------------------------------
07:01:09.955: Loaded scenes:
07:01:09.955: - scene 'Minecraft':
07:01:09.955: - source: 'MC' (game_capture)
07:01:09.955: - scene 'Runescape':
07:01:09.955: - source: 'Osrs' (window_capture)
07:01:09.955: - scene 'ROBLOX':
07:01:09.955: - source: 'RB' (game_capture)
07:01:09.955: ------------------------------------------------
07:01:20.429: ---------------------------------
07:01:20.429: video settings reset:
07:01:20.429: base resolution: 1920x1080
07:01:20.429: output resolution: 1920x1080
07:01:20.429: downscale filter: Bilinear
07:01:20.429: fps: 60/1
07:01:20.429: format: NV12
07:01:20.445: Settings changed (video)
07:01:20.445: ------------------------------------------------
07:01:31.227: Settings changed (outputs)
07:01:31.227: ------------------------------------------------
07:01:51.533: adding 21 milliseconds of audio buffering, total audio buffering is now 64 milliseconds
07:01:57.332: [game-capture: 'MC'] attempting to hook process: javaw.exe
07:01:57.335: [game-capture: 'MC'] using helper (compatibility hook)
07:01:57.361: [game-capture: 'MC'] hook not loaded yet, retrying..
07:02:00.349: [game-capture: 'MC'] attempting to hook process: javaw.exe
07:02:00.352: [game-capture: 'MC'] using helper (compatibility hook)
07:02:03.366: [game-capture: 'MC'] attempting to hook process: javaw.exe
07:02:03.370: [game-capture: 'MC'] using helper (compatibility hook)
07:02:03.385: [game-capture: 'MC'] hook not loaded yet, retrying..
07:02:06.383: [game-capture: 'MC'] attempting to hook process: javaw.exe
07:02:06.425: [game-capture: 'MC'] gl shared texture capture successful
07:02:06.432: [game-capture: 'MC'] shared texture capture successful
07:02:18.522: [game-capture: 'MC'] Hooked D3D9
07:02:19.932: adding 42 milliseconds of audio buffering, total audio buffering is now 106 milliseconds
07:02:53.170: Starting recording due to hotkey
07:02:53.171: ---------------------------------
07:02:53.171: [NVENC encoder: 'recording_h264'] settings:
07:02:53.171: rate_control: CBR
07:02:53.171: bitrate: 32000
07:02:53.171: cqp: 0
07:02:53.171: keyint: 250
07:02:53.171: preset: hp
07:02:53.171: profile: high
07:02:53.171: level: auto
07:02:53.171: width: 1920
07:02:53.171: height: 1080
07:02:53.171: 2-pass: false
07:02:53.171: b-frames: 2
07:02:53.171: GPU: 0
07:02:53.171:
07:02:53.821: ---------------------------------
07:02:53.822: [FFmpeg aac encoder: 'Track1'] bitrate: 160, channels: 2, channel_layout: 3
07:02:53.822:
07:02:53.824: [Media Foundation AAC: 'Track2']: encoder created
07:02:53.824: bitrate: 160
07:02:53.824: channels: 2
07:02:53.824: sample rate: 48000
07:02:53.824: bits-per-sample: 16
07:02:53.824:
07:02:53.845: ==== Recording Start ===============================================
07:02:53.845: [ffmpeg muxer: 'adv_file_output'] Writing file 'D:/Obs/2018-02-28 07-02-53.mp4'...
07:03:27.511: [game-capture: 'MC'] ------------------ gl capture freed ------------------
07:03:27.528: [game-capture: 'MC'] gl shared texture capture successful
07:03:27.531: [game-capture: 'MC'] shared texture capture successful
07:03:31.769: [game-capture: 'MC'] ------------------ gl capture freed ------------------
07:03:31.783: [game-capture: 'MC'] gl shared texture capture successful
07:03:31.798: [game-capture: 'MC'] shared texture capture successful
07:03:54.694: Stopping recording due to hotkey
07:03:55.069: [ffmpeg muxer: 'adv_file_output'] Output of file 'D:/Obs/2018-02-28 07-02-53.mp4' stopped
07:03:55.069: Output 'adv_file_output': stopping
07:03:55.069: Output 'adv_file_output': Total frames output: 3655
07:03:55.069: Output 'adv_file_output': Total drawn frames: 3174 (3674 attempted)
07:03:55.069: Output 'adv_file_output': Number of lagged frames due to rendering lag/stalls: 500 (13.6%)
07:03:55.069: ==== Recording Stop ================================================
This is my log.