Turtlepower, are you talking about the new South African English Siri voices as being the new ones you wanted to try out? If so, they do have weird issues. I like them though, so once those get fixed I may use them.
I am actually the opposite as you, I really do hate all compact voices. They sound raspy/rubbery and metallic to me, and some of them dont' have much clarity. If I have to use a compact voice, Samantha will do. But I always use the enhanced US Siri male voice.
In IOS 10, a lot of things changed with Siri voices to increase their naturalness. The intonation on the IOS 10 version was a little robotic for me, but I still wanted to try it in Voiceover. Strangely you couldn't use those versions with Voiceover though. When you downloaded the enhanced voices, it would fallback to the IOS 9 versions which, while being a little more lively sounding to me, they were of lower quality and had annoying artifacts I didn't like. I used it because it was my favorite at the time. I can't remember if compact Siri voices were available in IOS 10, but if I had to guess, I'd bet the old ones were still available.
But at some point the compact Siri voices seemed to have been removed. This was definitely the case in IOS 11, since when I started trying betas, the compact Siri voices weren't there. I'm pretty sure their removal coincides with Apple taking charge of development and using a different engine; Nuance previously just used the Vocalizer engine for the Siri voices which is why they could so easily work with the NvDA Vocalizer driver. Why they'd wait for IOS 11 to fully switch to the new engine is beyond me, but maybe it wasn't stable enough for Voiceover or something. That could well be, since when they first became available to Voiceover they indeed had really bad bugs. In any case, since siri voices use a different engine now, I doubt compact voices will exist anymore. Given Siri's purpose, I can understand why, though the way they quietly handled it is a little odd and unfortunately it directly affects people like you which isn't right. I'm not sure if new versions of IOS fall back on a compact voice if the enhanced ones aren't available, or if they just use Samantha or something now. But, if you were still using the old compact voices, or the old enhanced voices from IOS 9, those old versions will still work even on IOS 11 until you switch voices. At least that was the case for me, and it would make a lot of sense as to why you are only noticing this now. If you used the compact version for two years without a switch, nothing was forcing you not to use it. Once you switch though, the old versions that you were using are no longer available. Whether they are still on the phone is anyone's guess. They may be there, but then again they may not, you never know with Apple.
What pronunciation and robotic issues do you have with the enhanced US male? You said that it sounds fine in Siri but in Voiceover it is not to your liking? Can you detail this a little more? Can you specify the differences you hear, or provide a recording of how it sounds in Voiceover? There shouldn't be any differences, though I haven't done any real hard testing to confirm this. ON mine though, they are very similar and that is why I use them.
Make more of less, that way you won't make less of more!
If you like what you're reading, please give a thumbs-up.