What are you trying to do? Switch a camera source between landscape and portrait mode (90 degree change)? Flip an image horizontally or vertically (180 degree change)?
The simplest method to do either type in OBS would be to define a scene for each layout, with each scene containing the camera as a source. Then right-click the camera source in a scene and select "Transform" There will be options for rotate and flip. Resize the camera sources in the scene as desired.
During your presentation, select the scene that has the camera formatting you want.
You can assign hot keys to each scene to switch between them.