Oh dear, why didn't I see that thread before?
Note to all users: only use custom x264 encoder settings given by a source you personally feel trustworthy, or if you're bored and willing to experiment. We will not be able to help figure out problems with crazy encoder settings like in that thread mentioned above