Also - realize that you can stream to a provider (YouTube in your case) at a given resolution. The watching client device picks the resolution to watch at.
Have you logged into YouTube and confirmed the video is only 720p (vs that being the resolution a client device streams that video at) .... I know obvious/simple... but sometimes overlooked, so just checking the basics
And then there is base canvas and output resolution, which is/can be distinct from streaming resolution. And you can stream and locally record at different resolutions (if your hardware is sufficient for such a workload). So multiple settings to check. And then there is an automatic bandwidth throttle settings, if your WAN uplink isn't sufficient for sustained 1080p