x265 would only be supported for local recording, and not for streaming, unless Adobe added support for it to Flash, which they aren't very likely to do.
As for 10-bit color encoding, Adobe do say that it is supported by the Flash h.264 decoder, but they said the same thing about YUV 4:2:2 video, and that certainly wasn't the case. (It does not support YUV 4:4:4 either.)